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Abstract 

Human  errors  in  medical  device  use  account  for  a  large  portion  of  medical  errors. 
Most  of  these  errors  are  due  to  inappropriate  designs  for  user  interactions,  rather 
than  mechanical  failures.  Evaluating  and  predicting  patient  safety  in  medical 
device  use  is  critical  for  developing  interventions  to  reduce  such  errors  either  by 
redesigning  the  devices  or,  if  redesign  is  not  an  option,  by  training  the  users  on 
the  identified  trouble  spots  in  the  devices.  We  developed  two  methods  for 
evaluating  and  predicting  patient  safety  in  medical  devices  with  integral 
information  technology,  then  applied  and  tested  them  on  several  infusion  pumps. 
The  first  method  is  a  modified  discount-usability  method  called  heuristic 
evaluation.  The  method  was  used  to  evaluate  and  compare  the  safety  of  two  1- 
channel  volumetric  infusion  pumps.  The  results  show  that  heuristic  evaluation, 
when  modified  for  medical  devices,  is  a  useful,  efficient,  and  low-cost  method  for 
evaluating  patient  safety  features  of  medical  devices  through  the  identification  of 
usability  problems  and  their  severities.  The  second  method  is  an  extended 
hierarchical  task  analysis  (EHTA),  devised  to  predict  medical  errors  in  medical 
device  use.  EHTA  divides  the  task  space  between  the  external  world  of  the  device 
interface  and  the  internal  cognitive  world  of  the  user,  allowing  for  descriptive 
predictions  of  potential  user  errors  at  the  human  device  level.  Its  use  is 
demonstrated  in  the  analysis  of  two  infusion  pumps.  The  estimates  of  the 
likelihood  of  user  errors  with  the  two  pumps  are  consistent  with  the  corresponding 
reported  use  errors  in  the  Federal  Drug  Administration  (FDA)’s  Manufacturer  and 
User  Device  Experience  (MAUDE)  database,  thus  demonstrating  the  usefulness 
of  this  tool  for  predicting  medical  device  use  errors. 


Introduction 

Since  the  medical  error  report  from  the  Institute  of  Medicine  in  2000 1  and 
subsequent  increased  funding  for  research  on  medical  errors  from  U.S. 
government  and  private  institutions,  many  studies  on  medical  errors  have  been 
conducted  and  reported,  including  many  from  the  medical  infonnatics  and 
cognitive  science  communities.  ’  Our  research  has  focused  on  developing 
methods  for  evaluating  and  predicting  medical  errors  in  medical  device  use  from 
the  perspectives  of  cognitive  science  and  user  interface  design.4  6 
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Medical  device  use  errors  are  a  common  source  of  patient  injury  and  death. 
Numerous  research  reports,  medical  error  reports,  and  other  documents  show  a 
clear  link  between  usability  problems  and  user  error.7' 8  For  example,  a  physician 
treating  an  infant  with  oxygen  set  the  flow  knob  between  1  and  2  liters  per 
minute,  then  later  noticed  that  the  infant  was  not  receiving  any  oxygen.  Even 
though  the  knob  rotated  smoothly,  the  device  was  designed  to  deliver  oxygen  only 
when  the  knob  was  set  on  a  number,  not  between  numbers.  Adding  detents  to  the 
knob,  so  that  it  would  click  onto  a  number,  and  providing  visible  feedback  of  the 
rate  of  flow  could  have  greatly  decreased  the  chance  of  this  type  of  error.  This  is  a 
clear  example  of  medical  errors  caused  by  the  poor  design  of  the  device’s  user 
interface.  FDA  data  show  that  nearly  half  of  all  device  recalls  were  because  of 
poor  user  interface  design  of  the  products.9, 10  Other  research  also  suggests  that 
injuries  resulting  from  medical  device  use  errors  far  exceed  injuries  arising  from 
device  failures.11  A  poorly  designed  user  interface,  even  operated  by  a  well- 
trained  and  competent  user,  can  lead  to  errors  and  operating  inefficiencies.  In 
response,  the  FDA  has  revised  its  Good  Manufacturing  Practice  (GMP) 
regulations  to  include  specific  requirements  for  product  usability.9  It  has  also 
published  guidelines  for  interface  design  and  usability  testing10  and  produced  a 
continuing  education  article  that  specifically  covers  usability  issues.12  Some 
nonprofit  organizations  routinely  perform  and  publish  safety  evaluations  of 
medical  devices.  For  example,  Emergency  Care  Research  Institute  (ECRI) 
published  several  reports  on  the  evaluations  of  a  series  of  infusion  pumps.  These 
reports  are  typically  comprehensive,  covering  reviews  of  features,  functionalities, 
and  testing  results,  and  making  recommendations  based  on  assessment.  Human 
factors  are  also  considered  in  many  of  the  evaluations,  although  they  are  typically 
brief  and  at  a  high  level  without  much  detailed  infonnation.  The  methods  in  our 
current  article  can  potentially  expand  the  human  factors  evaluations  in  these  types 
of  reports,  providing  detailed  descriptions  of  usability  problems,  factors  causing 
the  problems,  suggested  solutions,  and  even  predictions  of  the  likelihood  of 
errors. 

Human  factors  engineering  is  a  discipline  that  seeks  to  design  devices, 
software,  and  systems  to  meet  the  needs,  capabilities,  and  limitations  of  the  users, 
rather  than  expecting  the  users  to  adapt  to  the  design.  A  complete  human  factors 
engineering  analysis  for  medical  devices  or  software  systems  includes  four  major 
components  (Figure  1):  user;  functional;  task;  and  representational  analyses.14 
User  analysis  is  the  process  of  identifying  the  characteristics  of  existing  and 
potential  users,  such  as  their  expertise  and  skills;  knowledge;  educational 
background;  cognitive  capacities  and  limitations;  perceptual  variations;  age- 
related  skills;  cultural  background;  personality;  time  available  for  learning  and 
training;  frequency  of  system  use;  and  so  on.  User  analysis  can  help  us  design 
systems  that  have  the  knowledge  and  information  structure  that  match  that  of  the 
users.  Functional  analysis  is  the  process  of  identifying  top-level  domain  structures 
and  goals  that  are  critical  for  the  system,  but  largely  independent  of 
implementations.  Task  analysis  is  the  process  of  identifying  the  procedures  and 
actions  to  be  carried  out,  and  the  information  to  be  processed  to  achieve  task 
goals.  Representational  analysis  is  the  process  of  identifying  an  appropriate 
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information-display  format  for  a  given  task  performed  by  a  specific  type  of  user 
such  that  the  interaction  between  the  users  and  the  system  is  as  direct  and 
transparent  as  possible.  With  direct  interaction  interfaces,  users  can  directly, 
completely,  and  efficiently  engage  in  the  primary  tasks  they  intend  to  perform,  not 
the  housekeeping  interface  tasks  that  are  barriers  between  users  and  systems.  The 
file  browser  in  Microsoft  Windows  uses  a  direct  interaction  interface  to  move, 
delete,  and  rename  files,  whereas  command  line  systems  (e.g.,  MS  DOS)  do  not. 


Figure  1.  Four  components  of  a  complete  human  factors  engineering  analysis 
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These  four  types  of  analyses,  when  combined  and  applied  to  a  single  product, 
can  reveal  the  full  range  of  usability  issues,  which  are  essential  for  an 
understanding  of  patient  safety  implications  of  the  product.  In  this  chapter,  we 
describe  two  methods  we  developed:4’ 15  heuristic  evaluation  and  extended 
hierarchical  task  analysis  (EHTA).  They  correspond  to  two  of  the  four  analyses  of 
human  factors  engineering — heuristic  evaluation  is  largely  a  type  of 
representational  analysis  and  extended  hierarchical  task  analysis  is  a  typical  task 
analysis. 

We  focus  on  heuristic  evaluation  because  it  has  been  shown  to  be  one  of  the 
most  cost-effective  methods  of  finding  usability  problems.  This  method  is  for  the 
evaluation  of  usability  problems  in  medical  devices.  Through  the  identification  of 
usability  problems,  we  can  indirectly  identify  medical  devices’  potential  trouble 
spots  that  are  likely  to  cause  medical  errors.  We  will  describe  the  usability 
heuristics,  the  scale  for  severity  rating  of  usability  problem,  and  the  procedure  of 
carrying  out  a  heuristic  evaluation.  Then  we  will  demonstrate  how  to  use  this 
method  by  applying  it  to  the  evaluation  of  two  1 -channel  volumetric  infusion 
pumps. 

Likewise,  we  focus  on  EHTA  because  it  can  reveal  the  deep  structures  of  a 
task  that  typically  show  the  root  causes  of  a  usability  problem.  This  is  a  method 
developed  to  predict  medical  errors  with  medical  devices.  EHTA  divides  the  task 
space  into  the  external  world  of  the  device  interface  and  the  internal  cognitive 
world  of  the  users,  allowing  for  descriptive  predictions  of  potential  user  errors  at 
the  human-device  level.  We  will  demonstrate  this  method  by  applying  it  to  the 
comparison  of  the  likelihood  of  medical  errors  with  two  infusion  pumps. 


Heuristic  evaluation 

Heuristic  evaluation  is  an  evaluation  technique  that  identifies  major  usability 
problems  of  a  product.16  19  This  method  has  become  extremely  popular  in  the 
realm  of  usability  evaluation  due  to  its  low  cost,  low  time  commitment,  and  ease 
of  application.18  This  technique  typically  requires  three  or  more  expert  usability 
evaluators  to  independently  apply  a  set  of  usability  heuristics  to  a  product, 
identify  violations  of  the  heuristics,  and  assess  the  severity  of  each  violation.  In 
general,  evaluators  can  conduct  the  evaluation  in  a  few  hours  with  minimal 
training.  For  a  complex  medical  device  such  as  the  infusion  pump,  the  evaluators 
should  also  have  some  clinical  knowledge  about  the  devices.  Evaluators  should 
already  possess  this  knowledge  or  can  obtain  it  through  training  at  a  level  that  is 
sufficient  for  the  understanding  and  use  of  the  device.  Ideally,  double  experts 
trained  in  both  usability  and  the  target  clinical  domain  should  be  the  evaluators.  In 
reality,  such  double  experts  are  in  short  supply  and  the  priority  is  typically  given 
to  the  expertise  of  usability  over  the  expertise  of  the  clinical  domain. 

Heuristic  evaluation  has  been  traditionally  used  to  evaluate  web  sites,  as  well 
as  desktop  software  applications,  and  it  is  typically  used  to  point  out  software 
interface  difficulties  to  be  addressed  in  the  design  process.  It  can  be  applied  to 
paper  or  electronic  mock-ups  or  prototypes,  as  well  as  completely  implemented 
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designs.  We  modified  this  method  to  address  three  issues  in  the  evaluation  of 
medical  devices.4'20  First,  we  used  it  to  discover  usability  problems  that  are  likely 
to  cause  medical  errors.  Second,  we  used  it  for  comparison  of  patient  safety 
features  of  alternative  medical  devices,  which  is  often  helpful  in  the  purchasing 
process  of  medical  devices.  Third,  we  tried  to  demonstrate  that  heuristic 
evaluation  is  a  good  tool  for  medical  device  manufacturers  to  improve  the  patient 
safety  features  of  their  products  during  the  design  and  redesign  processes.  We  will 
describe  this  method  and  its  application  in  the  evaluation  of  two  infusion  pumps. 

Usability  heuristics 

Nielsen1''  described  10  major  heuristics  that  should  be  followed  for  good  user 
interface  design.  Shneiderman2 1  also  described  8  golden  rules  that  all  good  user 
interface  designs  should  feature.  Based  on  the  work  of  Nielsen  and  Shneidennan, 
we  created  14  heuristics — aptly  named  Neilsen-Shneiderman  heuristics — with  a 
focus  on  medical  devices.4  More  details  of  these  heuristics  are  provided  by  Zhang, 
et  al;4  here,  we  only  give  a  top-level  description: 

1.  [ Consistency ]  Consistency  and  standards.  Users  should  not  have  to 
wonder  whether  different  words,  situations,  or  actions  mean  the  same 
thing.  Standards  and  conventions  in  product  design  should  be 
followed. 

2.  [ Visibility ]  Visibility  of  system  state.  Users  should  always  be 
informed  what  is  going  on  with  the  system  through  appropriate 
feedback  and  display  of  information. 

3.  [Match]  Match  between  system  and  world.  The  image  of  the  system 
perceived  by  users  should  match  the  model  the  users  have. 

4.  [ Minimalist ]  Minimalist.  Any  extraneous  information  is  a  distraction 
and  a  slowdown. 

5.  [Memory]  Minimize  memory  load.  Users  should  not  be  required  to 
memorize  a  lot  of  information  to  carry  out  tasks.  Memory  load  reduces 
users’  capacity  to  carry  out  the  main  tasks. 

6.  [Feedback]  Informative  feedback.  Users  should  be  given  prompt  and 
informative  feedback  about  their  actions. 

7.  [Flexibility]  Flexibility  and  efficiency.  Users  always  learn  and  users 
are  always  different.  Give  users  the  flexibility  of  creating 
customization  and  shortcuts  to  accelerate  their  performance. 

8.  [Message]  Good  error  messages.  The  messages  should  be 
informative  enough  such  that  users  can  understand  the  nature  of  errors, 
learn  from  errors,  and  recover  from  errors. 

9.  [Error]  Prevent  errors.  It  is  always  better  to  design  interfaces  that 
prevent  errors  from  happening  in  the  first  place. 

10.  [Closure]  Clear  closure.  Every  task  has  a  beginning  and  an  end.  Users 
should  be  clearly  notified  about  the  completion  of  a  task. 
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11.  [Undo]  Reversible  actions.  Users  should  be  allowed  to  recover  from 
errors.  Reversible  actions  also  encourage  exploratory  learning. 

12.  [Language]  Use  users’  language.  The  language  should  be  always 
presented  in  a  form  understandable  by  the  intended  users. 

13.  [ Control]  Users  in  control.  Don’t  give  users  the  impression  that  they 
are  controlled  by  the  systems. 

14.  [Document]  Help  and  documentation.  Always  provide  help  when 
needed. 

The  heuristics  are  used  to  check  the  interface  of  the  device  design.  If  a 
heuristic  is  violated,  it  is  given  a  severity  rating  based  on  the  following  scales:17 

0  =Not  a  usability  problem  at  all. 

1  =  Cosmetic  problem  only.  Need  not  be  fixed  unless  extra  time  is 

available. 

2  =  Minor  usability  problem.  Fixing  should  be  given  low  priority. 

3  =  Major  usability  problem.  Important  to  fix  and  should  be  given  high 

priority. 

4  =  Usability  catastrophe.  Imperative  to  fix  before  product  can  be  released. 

In  general  3-5  usability  experts  independently  evaluate  the  user  interface  of  a 
product  and  each  of  them  generates  a  separate  list  of  heuristic  violations 
according  to  the  14  heuristics  described  above.  A  single  usability  problem 
identified  by  an  evaluator  can  be  violations  of  multiple  heuristics,  which  means 
that  the  number  of  heuristic  violations  is  typically  more  than  the  number  of 
usability  problems  identified.  For  example,  the  oxygen  flow  control-knob 
problem  described  earlier  violates  consistency  and  standards  (most  devices  with 
smoothly  rotating  knobs  work  at  any  position);  visibility  of  system  status  (there 
was  no  indication  of  oxygen  flow);  match  between  the  system  and  the  world 
(users  expect  smoothly  rotating  knobs  to  work  at  all  positions);  and  prevent  errors 
(the  design  of  the  knob  and  lack  of  feedback  increase  the  chance  of  error).  In 
other  words,  the  oxygen  flow  control  is  one  usability  problem  corresponding  to 
four  heuristic  violations.  Once  each  evaluator  has  identified  potential  usability 
problems,  the  separate  lists  are  compiled  into  a  single  master  list.  The  master  list 
is  then  given  back  to  the  evaluators  who  independently  assess  the  severity  of  each 
violation.  The  ratings  from  the  individual  evaluators  are  then  averaged.  Once  the 
master  list  with  severity  ratings  is  generated,  it  can  be  sorted  in  different  ways, 
typically  by  places  of  occurrences  of  violations  and  by  severity  ratings. 

Heuristic  evaluation  is  relatively  easy  to  do,  even  for  those  not  trained  in 
usability.  We  have  found  that  two  to  three  hours  of  training,  combined  with  clear 
examples,  and  a  practice  evaluation  with  feedback,  is  often  sufficient  to  begin 
using  the  technique.  Typically  one  evaluator  can  only  catch  35  percent  of  the 
usability  problems,  but  3-5  evaluators  can  detect  60-75  percent.18  Heuristic 
evaluation  also  has  a  number  of  limitations.  It  does  not  indicate  the  elements  of 
the  interface  that  correctly  follow  usability  guidelines.  Nor  does  it  reveal  major 
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missing  functionality.  The  second  method  we  describe  in  this  chapter,  Extended 
Hierarchical  Task  Analysis,  partially  remedies  this  limitation. 

Heuristic  evaluation  of  two  1 -channel  infusion  pumps 

We  applied  the  14  heuristics  and  the  procedures  to  evaluate  the  usability 
problems  of  two  1 -channel  volumetric  infusion  pumps  from  two  different 
vendors.4  We  focused  on  identifying  usability  problems  that  might  be  potential 
triggers  for  medical  errors.  We  also  compared  the  two  infusion  pumps  in  their 
usability  and  patient  safety  features. 

Figure  2  shows  the  numbers  of  heuristic  violations  for  the  two  pumps  across 
the  14  heuristics.  For  Pump  1,  heuristics  were  violated  a  total  of  192  times. 
Consistency  and  visibility  were  the  two  most  frequently  violated  heuristics  (53 
and  28,  respectively).  Feedback  and  match  were  the  next  most  common  violations 
(22  and  21).  These  four  heuristics  account  for  64  percent  of  the  violations.  For 
Pump  2,  heuristics  were  violated  a  total  of  121  times.  Visibility  was  the  most 
frequently  violated  heuristic  (29  violations).  Memory  and  consistency  were  the 
next  most  common  violations  (19  and  17,  respectively).  These  three  heuristics 
comprised  54  percent  of  the  violations.  An  example  of  a  violation  of  the  visibility 
heuristic  would  be:  “When  the  ‘enter’  button  is  not  pressed,  after  entering  part  or 
all  of  the  value  for  ‘Rate’  and  ‘VTBI’  (Volume  to  be  Infused),  a  message  appears 
that  reads  ‘complete  entry.’”  It  is  not  clear  what  it  means.  A  better  phrasing  would 
be  “Press  ‘enter’  to  confirm  value.”  In  this  case,  users  are  apt  to  become  confused 
by  no  clear  delineation  as  to  what  action  would  come  next.  In  tenns  of  the 
quantity  of  heuristic  violations,  the  results  in  Figure  2  indicate  that  Pump  1  has 
more  usability  problems  and  thus  may  accordingly  have  a  higher  chance  of 
generating  medical  errors. 

Figure  3  summarizes  the  severity  of  the  problems  found  in  Pump  1  and  Pump 
2.  The  severity  ratings  were  divided  into  four  regions:  a  severity  rating  equal  or 
above  3.5  is  catastrophic;  a  severity  rating  equal  or  above  2.5  but  below  3.5  is 
major;  a  severity  rating  equal  or  above  1.5  but  below  1.5  is  minor;  and  a  severity 
rating  below  1.5  is  cosmetic.  For  Pump  1,  there  were  2  catastrophic,  38  major,  49 
minor,  and  zero  cosmetic  usability  problems.  For  Pump  2,  there  was  1 
catastrophic,  26  major,  26  minor,  and  zero  cosmetic  usability  problems.  In  tenns 
of  the  severity  of  usability  problems,  the  results  in  Figure  2  indicate  that  Pump  1 
has  a  larger  number  of  more  severe  usability  problems  and  thus  it  is  likely  to 
cause  more  medical  errors  than  Pump  2.  This  conclusion  is  consistent  with  the 
measure  in  terms  of  the  quantity  of  heuristic  violations:  Pump  1  has  more 
heuristic  violations  than  Pump  2.  In  short,  Pump  1  has  not  only  a  larger  number  of 
heuristic  violations  in  total,  but  also  a  larger  number  of  more  severe  heuristic 
violations. 

Summary  of  heuristic  evaluation 

As  a  discount  usability  technique,  heuristic  evaluation  is  easy  to  use  and 
master;  efficient;  effective;  and  useful.  It  can  be  used  to  identify  a  great  portion  of 
major  usability  problems  in  a  product  in  a  timely  manner  with  reasonable  cost. 
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Figure  2.  This  figure  shows  the  heuristic  violations  in  two  1 -channel  infusion  pumps 


Frequencies  of  heuristic  violations  by  categories 


14  heuristic  categories 


See  text  for  details. 

Reprinted  with  permission  from  Elsevier  from  Zhang  J,  Johnson  TR,  Patel  VL,  et  al.,  Using 
usability  heuristics  to  evaluate  patient  safety  of  medical  devices.  J.  Biomed  Inform.  2003; 
36(1-2):23-30. 


Figure  3.  Severity  ratings  of  usability  problems  for  the  two  1  -channel  infusion  pumps 
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Reprinted  with  permission  from  Elsevier  from  Zhang  J,  Johnson  TR,  Patel  VL,  et  al.,  Using 
usability  heuristics  to  evaluate  patient  safety  of  medical  devices.  J  Biomed  Inform.  2003; 
36(1-2):23-30. 

Human  errors  in  medical  device  use  are  largely  due  to  interface  design  problems 
that  can  be  potentially  addressed  through  user-centered  design.  Since  the  quantity 
and  severity  of  usability  problems  are  usually  correlated  with  the  frequency  of 
medical  errors,  heuristic  evaluation  is  a  method  for  indirectly  assessing  patient 
safety  features  in  medical  devices.  Although  it  is  limited  in  its  scope  of  coverage 
of  the  full  range  of  patient  safety  related  features  in  medical  devices,  it  is  a 
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practical  tool  that  should  be  adopted  by  medical  device  manufacturers  for  the 
design  and  modification  of  medical  devices,  and  by  health  care  institutions  for  the 
evaluation  of  medical  devices.  Usability  engineering  has  become  a  niche  industry 
with  many  professionals  trained  in  human  factors  and  related  disciplines.  Medical 
device  manufacturers  should  routinely  use  the  services  offered  by  the  usability 
industry  or  perform  usability  evaluations  in-house  by  hiring  usability  specialists. 
The  second  method  we  describe  below  will  complement  the  heuristic  evaluations 
to  address  a  different  set  of  issues  in  medical  errors. 


Extended  hierarchical  task  analysis 

The  heuristic  evaluation  method  we  described  in  the  last  section  is  mainly  for 
the  evaluation  of  usability  problems  that  may  potentially  cause  medical  errors.  In 
this  section,  we  describe  an  extended  hierarchical  task  analysis  method  with  a 
focus  on  predicting  medical  errors  in  medical  device  use.3  While  the  prospect  of  a 
bulletproof  method  for  predicting  human  error  given  an  interface  and  task  still 
seems  distant,  the  current  state  of  theory  and  research  in  the  field  of  human- 
computer  interaction  (HCI)  and  cognitive  science  seems  sufficient  to  provide 
much  improvement  over  current  techniques.  These  techniques  will  be  priceless  if 
successfully  applied  in  the  medical  arena  where  human  lives  are  consistently  at 
stake.  This  work  represents  one  such  effort. 

Hierarchical  task  analysis  and  its  extension 

Hierarchical  task  analysis  (HTA),  one  of  the  most  widely  used  forms  of  task 
analysis,  involves  describing  a  task  as  a  hierarchy  of  tasks  and  subtasks.22  The 
first  three  columns  of  Figure  4  show  a  traditional  hierarchical  task  analysis.  First, 
major  tasks  are  identified.  Then  the  sub  tasks  of  each  major  task  are  identified,  and 
sub  tasks  of  the  sub  tasks  are  identified  until  meaningful  sub  tasks  are  exhausted. 
This  process  establishes  a  hierarchy.  Each  task  and  subtask  in  the  hierarchy  is 
assigned  a  code  that  indicates  the  level  of  the  task  and  the  sequence  in  the  task.  In 
the  example  in  Figure  4,  the  first  level  tasks  are  coded  as  l.xx,  2.xx,  etc.;  the 
second  level  tasks  are  coded  as  l.lx,  1.2x,  etc.;  and  so  on.  By  doing  a  traditional 
HTA,  the  structures  of  the  tasks  and  the  interrelations  of  tasks  and  subtasks  can  be 
explicitly  described.  This  is  a  very  useful  process  for  the  understanding  and 
design  of  any  user  interfaces. 

However,  the  traditional  HTA  is  limited.  It  does  not  provide  enough  details 
about  the  cognitive  processes  and  information  processing  that  are  typically  crucial 
for  user  interface  design.  In  our  extended  hierarchical  task  analysis  (EHTA),  we 
added  a  few  more  steps  that  are  especially  useful  for  the  prediction  of  medical 
errors.  In  Figure  4,  the  columns  after  the  first  three  are  added  steps  in  EHTA. 
These  added  steps  are  described  below. 

The  physical  features  of  a  device  are  added  for  each  step  identified  in  the 
traditional  HTA.  These  help  determine  the  representations  that  are  analyzed  next. 
For  example,  a  given  state  might  indicate  that  the  device  is  powered  on.  More 
detailed  information  can  be  provided,  such  as  auditory  cues,  display  messages, 


331 


Advances  in  Patient  Safety:  Vol.  2 


soft  key  information,  etc.  Previous  state  and  display  information  are  also 
recorded,  since  these  dictate  the  context  or  priming  the  user  is  given  as  they 
proceed  to  the  next  step.  This  information  is  used  to  decide  what  must  be  done  on 
the  subsequent  screen,  and  thus  has  a  great  effect  on  the  user’s  behavior. 

The  next  step  is  to  analyze  whether  the  information  needed  to  carry  out  each 
step  in  a  task  is  represented  as  internal  or  external,  using  the  distributed 
representation  theory  developed  by  Zhang  and  Norman.2  ’  Internal  representation 
is  information  stored  in  users’  memory,  whereas  external  representation  is 
available  on  the  displays  or  other  remote  medium  or  devices.  In  the  infusion  pump 
example,  internal  representation  is  declarative  knowledge  or  mental  operators 
needed  by  a  user  to  complete  that  step  of  the  task,  and  external  representation  is 
the  information  available  from  the  columns  for  soft  keys,  auditory  warnings, 
display  messages,  etc.,  needed  by  the  user  to  execute  that  step.  According  to 
Zhang  and  Norman,  external  representation — where  the  information  is  present  on 
the  device  itself  rather  than  in  the  head  of  the  operator — is  thought  to  drastically 
reduce  or  eliminate  human  error.  Internal  representation  depends  on  cognitive 
processes  for  retrieval,  which  is  error  prone,  but  external  representation  relies  on 
more  efficient  perceptual  processes,  hence  mitigating  cognitive  load.  In  order  to 
accurately  evaluate  this  representation,  it  is  necessary  to  consider  all  information 
presented  in  this  manner.  This  includes  auditory  warnings,  text  messages,  LEDs, 
physical  controls,  etc.  Research  shows  that  the  specific  wording  and  presentation 
of  labels  and  feedback  messages  have  a  great  impact  on  the  actions  users  decide 
to  take.24  Listing  all  state  and  transitional  information  makes  it  possible  for  the 
evaluator  to  identify  deficiencies  in  the  external  task  space. 

The  next  step  is  to  analyze  the  error  affordance  for  each  step  of  a  task.  Error 
affordance  is  detennined  by  a  set  of  task  characteristics  found  by  psychological 
research  to  cause  error.25,26  For  example,  an  isolated  step  in  a  task  affords  error, 
such  as  the  step  of  entering  a  hard  key  to  find  the  “secondary”  mode  when 
programming  an  intravenous  piggyback  (IVPB)  on  an  infusion  pump.  In  general, 
the  more  action  opportunities  at  each  step,  the  higher  the  error  affordance,  i.e.,  the 
more  likely  a  wrong  action  is  selected.  Error  affordance  is  also  associated  with 
internal  and  external  representations:  the  more  information  internal,  the  higher  the 
error  affordance  because  internal  representations  are  more  error  prone.  In  the  next 
subsection,  we  demonstrate  EHTA  by  using  it  to  analyze  and  compare  two 
infusion  pumps. 
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An  example  of  extended  hierarchical  task  analysis 

To  make  the  methodology  of  EHTA  more  concrete,  we  did  a  sample  analysis 
using  1 -channel  and  3 -channel  pumps  from  Manufacturer  A  and  Manufacturer  B.5 
Pumps  from  Manufacturer  A  are  widely  used,  well  recognized  and  touted  for  their 
human-centered  design;  the  pumps  are  claimed  to  have  several  human  factors 
improvements  over  the  previous  technology.  Nonetheless,  a  quick  search  through 
FDA  reports  makes  it  clear  that  they  fall  short  of  being  problem-free.  Pumps  from 
Manufacturer  B  are  a  competing  product  in  the  market. 

Two  1 -channel  and  two  3-channel  pumps  from  the  two  manufacturers  were 
analyzed  on  two  different  tasks  to  show  their  intrinsic  differences.  Task  1  was  to 
infuse  1,000  ml  normal  saline  at  125  ml  per  hour.  Task  2  was  to  infuse  1,000  ml 
of  normal  saline  at  125  ml  per  hour  as  a  continuous  infusion;  give  lgm 
Ceftaxidime  (Fortaz)  IVPB  every  8  hours;  and  administer  over  30  minutes. 

In  the  left  panel  of  Figure  5,  the  first  two  columns  show  the  average  numbers 
of  steps  needed  to  perform  Task  1  and  Task  2;  the  second  two  columns  the 
numbers  of  error  affordances  for  1 -channel  Pump  A  and  Pump  B.  The  third  two 
columns  show  the  actual  usage  error  data  from  FDA’s  MAUDE  database  (2001 
and  2002).  It  shows  that  Pump  A  requires  more  steps  than  Pump  B,  and  Pump  A 
has  more  error  affordances  than  Pump  B.  In  addition,  this  pattern  is  consistent 
with  the  FDA  data  on  the  actual  error  cases  for  Pump  A  and  Pump  B.  The  right 
panel  of  Figure  5  is  for  the  3-channel  pumps  by  Manufacturers  A  and  B.  The 
results  are  similar  to  those  for  the  1 -channel  pumps.  Caution  should  be  exercised 
here:  there  is  the  possibility  of  poor  correlation  between  the  data  from  EHTA  and 
the  data  in  the  MAUDE  database  because  the  exact  market  share  information  of 
Pump  A  and  Pump  B  is  unknown  to  us.  At  the  local  Houston  market  where  this 
study  was  conducted,  it  appears  that  Pump  A  was  more  popular  than  Pump  B. 

This  analysis  example  shows  that  EHTA  is  a  reasonably  good  method  for 
predicting  the  likelihood  of  medical  errors  for  a  specific  medical  device.  It 


Figure  5.  Comparison  of  infusion  pumps  by  extended  hierarchical  task  analysis  and 
data  from  FDA  database.  See  text  for  details. 

Single  Channel  Triple  Channel 
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established  a  strong  correlation  between  error  affordance  and  error  frequency. 
However,  our  method  is  not  foolproof  for  predicting  human  errors  given  an 
interface.  Furthermore,  it  remains  highly  dependent  upon  the  skill  of  the 
evaluator.  Even  so,  it  at  least  draws  ah  major  cognitive  factors  into  consideration, 
extending  present  techniques  and  traditional  treatments  of  human  error  to  a  deeper 
cognitive  level.  And  our  methodology  is  practical;  as  demonstrated,  simply 
tabulating  the  external  features  of  an  interface  side-by-side  with  the  steps  required 
can  allow  for  quick  visual  comparisons  of  different  pump  models. 


Conclusion 

In  this  paper,  we  described  two  human  factors  engineering  methods  for  the 
evaluation  and  prediction  of  medical  errors  in  medical  device  use.  Human  errors 
in  medical  device  use  account  for  a  large  proportion  of  medical  errors.  Most  of 
these  errors  are  due  to  inappropriate  designs  for  user  interactions,  rather  than 
mechanical  failures.  Evaluating  and  predicting  patient  safety  in  medical  device 
use  is  critical  for  developing  interventions  to  reduce  such  errors  either  by 
redesigning  the  devices  or,  if  redesign  is  not  an  option,  then  by  training  the  users 
on  the  identified  trouble  spots  in  the  devices.  We  have  shown  initial  success  of 
these  two  methods  for  infusion  pumps.  We  believe  that  these  two  methods,  with 
little  or  no  modification,  can  be  applied  to  health  information  systems  as  well  as 
other  medical  devices  for  a  quick  and  inexpensive  evaluation.  With  further 
refinement  and  validation  of  the  methods,  they  have  the  potential  to  become  solid 
tools  for  manufacturers,  purchasers,  and  consumers  to  evaluate  patient  safety 
issues  in  various  health  related  products. 
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